Import¶
In [ ]:
import warnings
from sklearn.exceptions import UndefinedMetricWarning
warnings.filterwarnings("ignore", category=UndefinedMetricWarning)
warnings.filterwarnings("ignore", category=UserWarning)
warnings.filterwarnings("ignore", category=FutureWarning)
import plotly
plotly.offline.init_notebook_mode()
In [ ]:
from file_py.run_log_parser import RunLogParser
from file_py.csv_preprocessing_scaler import CsvPreprocessingScaler
from file_py.plots import Plots
from file_py.utils import MarkdownHelper
from file_py.attack_log_unification import AttackLogUnification
from file_py.stat_severity import StatSeverity
from file_py.attack_pattern_analyzer import AttackPatternAnalyzer
from file_py.signatures_patterns import SignaturePatterns
from file_py.signature_stats_calculator import SignatureStatsCalculator
from file_py.sigma_rule_analysis import SigmaRuleAnalysis
from file_py.plots_single_attack import PlotsSingleAttack
from file_py.correlation_matrix_plots import CorrelationMatrixPlots
from file_py.preprocessing_train_test_split import PreprocessingTrainTestSplit
from file_py.initial_training import InitialTraining
from file_py.hyperparameter_tuning import HyperparameterTuning
from file_py.advanced_models import AdvancedModels
from file_py.deep_learning_model import DeepLearningModel
from file_py.model_evaluator import ModelEvaluator
CARICAMENTO FILE¶
Sostituire il percorso dei file attuali con il percorso dei file di interesse qui:
In [ ]:
# FILE CONTENENTE I LOG
df = CsvPreprocessingScaler.read_csv_file("file_csv/Log_splunk_combined.csv")
# FILE CON LE DATE DI INIZIO E FINE DEGLI ATTACCHI
files = ['file_csv/attackLog_combined.csv']
Preprocessing¶
In [ ]:
df_raw = CsvPreprocessingScaler.RawPreprocessing(df)
df_Le = CsvPreprocessingScaler.LEPreprocessing(df)
df_OH = CsvPreprocessingScaler.OhePreprocessing(df)
In [ ]:
df_std_LE = CsvPreprocessingScaler.stdScaler(CsvPreprocessingScaler.LEPreprocessing(df))
df_std_OH = CsvPreprocessingScaler.stdScaler(CsvPreprocessingScaler.OhePreprocessing(df))
Test¶
In [ ]:
attack_log_path = AttackLogUnification.attack_log_together( files,'file_csv/attackLog_combined.csv')
In [ ]:
result_df_Le = RunLogParser.process_attacks(attack_log_path, CsvPreprocessingScaler.stdScaler(CsvPreprocessingScaler.LEPreprocessing(df)))
result_df_OH = RunLogParser.process_attacks(attack_log_path, CsvPreprocessingScaler.stdScaler(CsvPreprocessingScaler.OhePreprocessing(df)))
result_df_Raw = RunLogParser.process_attacks(attack_log_path, CsvPreprocessingScaler.RawPreprocessing(df))
Graphic Analysis of Attacks¶
In [ ]:
Plots.plot_cake_attack(result_df_Raw)
In [ ]:
Plots.plot_top_10_signatures(result_df_Raw)
Out[ ]:
Qui si può notare come generalmente le regole scattate più volte sono anche quelle che hanno effettivamente risposto a più attachi e che sono scattate a vuoto più volte.
In [ ]:
Plots.plot_precision_recall(result_df_Raw)
Il primo grafico mostra la precisione di ciascuna regola, cioè la proporzione di attivazioni corrette rispetto al totale delle sue attivazioni.
Una precisione più alta indica che la regola è più accurata nel rilevare veri attacchi.
Il secondo grafico mostra il recall, cioè la proporzione di attacchi reali rilevati dalla regola rispetto al totale degli attacchi reali.
Un recall più alto indica che la regola è più efficace nel rilevare tutti gli attacchi possibili.
In [ ]:
Plots.plot_distributions(result_df_Raw)
In [ ]:
Plots.plot_value_counts_per_unique(result_df_Raw)
In [ ]:
variables = MarkdownHelper.create_value_counts_variables(result_df_Raw)
MarkdownHelper.display_value_counts_text(variables)
Grazie a questo grafico invece possiamo giungere ad una serie di conclusioni.
Su 103 regole diverse:
quelle scattate in risposta ad ALMENO un attacco reale sono 88. Di queste:
- 27 si sono attivate più volte per non-attacchi rispetto che per gli attacchi. (regole generiche)
- 20 si sono attivate lo stesso numero di volte per attacchi e non-attacchi.
- 41 si sono attivate più volte in risposta ad attacchi rispetto che a non-attacchi (regole specifiche).
quelle scattate senza rispondere mai ad attacchi sono 15.
Si tratta di: ['proc-start-copy-from-volumeshadowcopy-via-cmd.exe', 'proc-start-service-registry-key-deleted-via-reg.exe', 'proc-start-pua-suspicious-activedirectory-enumeration-via-adfind.exe', 'proc-start-suspicious-mshta-child-process', 'proc-start-scheduled-task-executing-encoded-payload-from-registry', 'proc-start-run-powershell-script-from-ads', 'proc-start-file-in-suspicious-location-encoded-to-base64-via-certutil.exe', 'proc-start-network-reconnaissance-activity', 'proc-start-process-memory-dump-via-rdrleakdiag.exe', 'proc-start-renamed-createdump-utility-execution', 'proc-start-potential-persistence-via-logon-scripts-commandline', 'proc-start-suspicious-process-patterns-ntds.dit-exfil', 'proc-start-lolbas-encode-decode', 'proc-start-whoami.exe-execution-anomaly', 'proc-start-lsass-process-reconnaissance-via-findstr.exe']
Analysis of Severity per Attacks¶
In [ ]:
event_df = RunLogParser.create_event_df(attack_log_path, result_df_Raw)
Creazione del df event_df con le nuove colonne severity_max, _min, _mean
Grafici¶
In [ ]:
StatSeverity.plot_stat_severity(event_df)
In questo grafico vediamo, per ciascun attacco presente nel dataset, quali sono le loro criticità massime, minime e medie.
In [ ]:
analyzer = AttackPatternAnalyzer(event_df)
In [ ]:
# SCEGLIERE UN VALORE PER LA SEVERITY DELLE REGOLE DA CONSIDERARE
severity_value=73
# SCEGLIERE IL NUMERO DI ATTACCHI DA CONSIDERARE PRIMA DELLE REGOLE AVENTI LA SEVERITY SCELTA
num_attacks=10
In [ ]:
analyzer.pattern_before_attack(num_attacks=num_attacks, severity_value=severity_value)
In questi grafici prendiamo in considerazione gli attacchi precedenti a tutti gli attacchi che hanno una certa criticità media e visualizziamo tutti i valori di "RuleAnnotation.mitre_attack.id", "signature", "EventType", "tag", "severity_id" corrispondenti.
Per scegliere quanti attacchi prima di quelli che ci interessano vogliamo considerare basta modificare la variabile "num_attacks" e assegnarle il numero che vogliamo,
mentre per scegliere il valore della criticità media che ci interessa si deve modificare la variabile "severity_value".
Verranno presi in considerazione tutti gli attacchi con criticità media compresa tra 2.5 prima e 2.5 dopo del valore assegnato a "severity_value".
Robustezza regole¶
In [ ]:
signature_stats = SignatureStatsCalculator.create_signature_stats(event_df, result_df_Raw)
signature_stats
Out[ ]:
| signature | Indice_Diff | Media_Differenza_Severity_min | Media_Differenza_Severity_mean | Media_Differenza_Severity_max | N_Max_Sev_Diff_15 | N_Attacchi_Non_rilevati | |
|---|---|---|---|---|---|---|---|
| 0 | proc-start-dumping-of-sensitive-hives-via-reg.exe | 0.000421 | 0.000000 | 0.068447 | 0.000000 | 0 | 0 |
| 1 | proc-start-copying-sensitive-files-with-creden... | 0.001330 | 0.000000 | 0.059737 | 0.000000 | 0 | 0 |
| 2 | suspicious-volume-shadow-copy-vss_ps.dll-load | 0.000882 | 0.000000 | 0.005974 | 0.000000 | 0 | 0 |
| 3 | net-connect-80-443-non-browser | 0.054409 | -9.537037 | -2.637203 | 0.000000 | 0 | 9 |
| 4 | proc-start-malicious-powershell-commandlets-pr... | 0.011164 | 0.000000 | 0.325955 | 0.179856 | 1 | 1 |
| ... | ... | ... | ... | ... | ... | ... | ... |
| 98 | proc-start-renamed-createdump-utility-execution | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0 | 0 |
| 99 | proc-start-process-memory-dump-via-rdrleakdiag... | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0 | 0 |
| 100 | proc-start-abused-debug-privilege-by-arbitrary... | 0.000052 | 0.000000 | 0.000000 | 0.000000 | 0 | 0 |
| 101 | proc-start-potential-cobaltstrike-process-patt... | 0.000034 | 0.000000 | 0.000000 | 0.000000 | 0 | 0 |
| 102 | proc-start-potential-meterpreter/cobaltstrike-... | 0.000072 | 0.000000 | 0.000000 | 0.000000 | 0 | 0 |
103 rows × 7 columns
signature_stats è un dataset in cui possiamo vedere per ogni regola se venisse rimossa quali cambiamenti di severity apporterebbe al dataset dei log e se ci dovessero essere degli attacchi che non vengono rilevati.
In [ ]:
analysis = SigmaRuleAnalysis(signature_stats)
analysis.plots_sigma_rule_analysis()
Graphic Analysis of Attacks for Chosen Rule¶
In [ ]:
# SCEGLIERE LA REGOLA CHE SI VUOLE ANALIZZARE
regola_scelta = 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'
In [ ]:
PlotsSingleAttack.analyze_rule_activations(result_df_Raw, regola_scelta)
In questi grafici in base alla regola che si vuole analizzare possiamo visualizzare:
- la frequenza delle attivazioni delle regole (attacchi e non attacchi) suddivise in intervalli di 5 minuti;
- gli attacchi e i non-attacchi in base a:
- RuleAnnotation.mitre_attack.id
- EventType,
- severity,
- tag
- parent_process_id,
- process_id
In [ ]:
# SCEGLIERE IL NUMERO DI EVENTI PRECEDENTI ALLA REGOLA CHE SI VOGLIONO ANALIZZARE
eventi_da_considerare = 5
In [ ]:
PlotsSingleAttack.patterns_before_activation(result_df_Raw, regola_scelta, eventi_da_considerare)
In questi grafici vediamo quali sono rispettivamente le regole, gli attacchi, gli EventType, i tag, i parent_process, i process e le severity degli eventi subito prima delle prime attivazioni della regola scelta.
Il numero di eventi da considerare lo scegliamo affidando alla variabile elementi_da_considerare il numero che vogliamo.
Con "prima attivazione di una regola" si intende quando almeno un elemento delle colonne signature, RuleAnnotation.mitre_attack.id, EventType, tag, severity_id, parent_process_id o process_id (non sono considerate solo le colonne _time e corrisponde_ad_attacco) di un evento differisce da quello precedente.
Patterns¶
In [ ]:
signature_patterns = SignaturePatterns.recognize_signatures_patterns(result_df_Raw)
signature_patterns
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 424
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 389
Pattern: ('proc-start-reg-add-suspicious-paths', 'proc-start-reg-add-suspicious-paths'), Frequenza: 30
Pattern: ('proc-start-reg-add-suspicious-paths', 'proc-start-reg-add-suspicious-paths', 'proc-start-reg-add-suspicious-paths'), Frequenza: 28
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern'), Frequenza: 27
Pattern: ('proc-start-reg-add-suspicious-paths', 'proc-start-reg-add-suspicious-paths', 'proc-start-reg-add-suspicious-paths', 'proc-start-reg-add-suspicious-paths'), Frequenza: 26
Pattern: ('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-powershell-download-and-execution-cradles'), Frequenza: 23
Pattern: ('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-powershell-download-and-execution-cradles'), Frequenza: 23
Pattern: ('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-powershell-download-and-execution-cradles'), Frequenza: 23
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser'), Frequenza: 20
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser'), Frequenza: 20
Pattern: ('net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 17
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser'), Frequenza: 17
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-program-names'), Frequenza: 17
Pattern: ('net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 15
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-powershell-download-and-execution-cradles'), Frequenza: 13
Pattern: ('proc-start-suspicious-process-parents', 'proc-start-suspicious-process-parents'), Frequenza: 13
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 11
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 11
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 11
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 11
Pattern: ('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-program-names'), Frequenza: 11
Pattern: ('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-program-names'), Frequenza: 11
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-powershell-download-and-execution-cradles'), Frequenza: 11
Pattern: ('proc-start-suspicious-process-parents', 'proc-start-suspicious-process-parents', 'proc-start-suspicious-process-parents'), Frequenza: 11
Pattern: ('net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 10
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 10
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-powershell-download-and-execution-cradles'), Frequenza: 10
Pattern: ('reg-value-write-cert-change', 'reg-value-write-cert-change', 'reg-value-write-cert-change', 'reg-value-write-cert-change'), Frequenza: 10
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern'), Frequenza: 9
Pattern: ('net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 9
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 9
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 9
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 9
Pattern: ('proc-start-suspicious-process-parents', 'proc-start-suspicious-process-parents', 'proc-start-suspicious-process-parents', 'proc-start-suspicious-process-parents'), Frequenza: 9
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'net-connect-Windows-processes'), Frequenza: 8
Pattern: ('net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 8
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'net-connect-Windows-processes'), Frequenza: 8
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-80-443-non-browser'), Frequenza: 8
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser', 'net-connect-80-443-non-browser'), Frequenza: 7
Pattern: ('net-connect-Windows-processes', 'net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 7
Pattern: ('net-connect-Windows-processes', 'net-connect-Windows-processes', 'net-connect-80-443-non-browser'), Frequenza: 7
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser', 'net-connect-80-443-non-browser'), Frequenza: 7
Pattern: ('net-connect-Windows-processes', 'net-connect-Windows-processes', 'net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 7
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-80-443-non-browser'), Frequenza: 7
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern'), Frequenza: 7
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 7
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 7
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 7
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-powershell-download-and-execution-cradles'), Frequenza: 7
Pattern: ('net-connect-80-443-non-browser', 'net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 6
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-Windows-processes', 'net-connect-Windows-processes'), Frequenza: 6
Pattern: ('net-connect-Windows-processes', 'net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 6
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-Windows-processes', 'net-connect-Windows-processes', 'net-connect-Windows-processes'), Frequenza: 6
Pattern: ('net-connect-Windows-processes', 'net-connect-80-443-non-browser', 'net-connect-Windows-processes', 'net-connect-Windows-processes'), Frequenza: 6
Pattern: ('proc-start-dns-exfiltration-nslookup', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 6
Pattern: ('proc-start-suspicious-program-names', 'proc-start-hacktool-sharpview-execution'), Frequenza: 6
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-program-names', 'proc-start-hacktool-sharpview-execution'), Frequenza: 6
Pattern: ('proc-start-suspicious-program-names', 'proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution'), Frequenza: 6
Pattern: ('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-program-names', 'proc-start-hacktool-sharpview-execution'), Frequenza: 6
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-program-names', 'proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution'), Frequenza: 6
Pattern: ('proc-start-suspicious-program-names', 'proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution'), Frequenza: 6
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 5
Pattern: ('net-connect-80-443-non-browser', 'net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 5
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'net-connect-Windows-processes', 'net-connect-Windows-processes'), Frequenza: 5
Pattern: ('net-connect-Windows-processes', 'net-connect-Windows-processes', 'net-connect-Windows-processes', 'net-connect-80-443-non-browser'), Frequenza: 5
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 5
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 5
Pattern: ('proc-start-suspicious-program-names', 'proc-start-suspicious-powershell-download-and-execute-pattern'), Frequenza: 5
Pattern: ('proc-start-suspicious-program-names', 'proc-start-powershell-download-and-execution-cradles'), Frequenza: 5
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-potentially-suspicious-powershell-child-processes'), Frequenza: 5
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-potentially-suspicious-powershell-child-processes'), Frequenza: 5
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 5
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 5
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-suspicious-target-names'), Frequenza: 4
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-suspicious-target-names'), Frequenza: 4
Pattern: ('potential-system-dll-sideloading-from-non-system-locations', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 4
Pattern: ('potential-system-dll-sideloading-from-non-system-locations', 'potential-system-dll-sideloading-from-non-system-locations', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 4
Pattern: ('potential-system-dll-sideloading-from-non-system-locations', 'potential-system-dll-sideloading-from-non-system-locations', 'potential-system-dll-sideloading-from-non-system-locations', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 4
Pattern: ('net-connect-Windows-processes', 'net-connect-Windows-processes', 'net-connect-80-443-non-browser', 'net-connect-Windows-processes'), Frequenza: 4
Pattern: ('proc-start-suspicious-program-names', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 4
Pattern: ('proc-start-suspicious-program-names', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 4
Pattern: ('proc-start-suspicious-program-names', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 4
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-80-443-non-browser', 'net-connect-Windows-processes'), Frequenza: 4
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-80-443-non-browser', 'net-connect-Windows-processes'), Frequenza: 4
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 4
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 4
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 4
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-program-names', 'proc-start-suspicious-powershell-download-and-execute-pattern'), Frequenza: 4
Pattern: ('proc-start-hacktool-sharpview-execution', 'proc-start-suspicious-program-names'), Frequenza: 4
Pattern: ('proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution', 'proc-start-suspicious-program-names'), Frequenza: 4
Pattern: ('proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution', 'proc-start-suspicious-program-names'), Frequenza: 4
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-Windows-processes'), Frequenza: 4
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-Windows-processes'), Frequenza: 4
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-Windows-processes', 'net-connect-80-443-non-browser'), Frequenza: 4
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-Windows-processes', 'net-connect-80-443-non-browser'), Frequenza: 4
Pattern: ('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern'), Frequenza: 4
Pattern: ('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern'), Frequenza: 4
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-suspicious-target-names', 'net-connect-Windows-processes'), Frequenza: 3
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-suspicious-target-names', 'net-connect-Windows-processes'), Frequenza: 3
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-suspicious-target-names', 'net-connect-Windows-processes', 'net-connect-80-443-non-browser'), Frequenza: 3
Pattern: ('net-connect-suspicious-sources', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('net-connect-80-443-non-browser', 'net-connect-suspicious-sources', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('net-connect-suspicious-sources', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('net-connect-80-443-non-browser', 'net-connect-suspicious-sources', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('net-connect-suspicious-sources', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'potential-system-dll-sideloading-from-non-system-locations'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'potential-system-dll-sideloading-from-non-system-locations'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'potential-system-dll-sideloading-from-non-system-locations', 'potential-system-dll-sideloading-from-non-system-locations'), Frequenza: 3
Pattern: ('potential-system-dll-sideloading-from-non-system-locations', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser', 'net-connect-80-443-non-browser', 'net-connect-80-443-non-browser'), Frequenza: 3
Pattern: ('net-connect-80-443-non-browser', 'net-connect-80-443-non-browser', 'net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'potential-system-dll-sideloading-from-non-system-locations'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'potential-system-dll-sideloading-from-non-system-locations', 'potential-system-dll-sideloading-from-non-system-locations'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'potential-system-dll-sideloading-from-non-system-locations', 'potential-system-dll-sideloading-from-non-system-locations', 'potential-system-dll-sideloading-from-non-system-locations'), Frequenza: 3
Pattern: ('potential-system-dll-sideloading-from-non-system-locations', 'potential-system-dll-sideloading-from-non-system-locations', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser', 'net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 3
Pattern: ('net-connect-80-443-non-browser', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 3
Pattern: ('net-connect-80-443-non-browser', 'net-connect-suspicious-sources', 'net-connect-80-443-non-browser', 'net-connect-80-443-non-browser'), Frequenza: 3
Pattern: ('proc-start-suspicious-double-extension-file-execution', 'proc-start-dir-user-writeable'), Frequenza: 3
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-80-443-non-browser', 'net-connect-Windows-processes', 'net-connect-suspicious-target-names'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'reg-value-write-cert-change', 'reg-value-write-cert-change'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'reg-value-write-cert-change', 'reg-value-write-cert-change', 'reg-value-write-cert-change'), Frequenza: 3
Pattern: ('proc-start-dir-user-writeable', 'reg-value-write-cert-change'), Frequenza: 3
Pattern: ('reg-value-write-cert-change', 'proc-start-dir-user-writeable'), Frequenza: 3
Pattern: ('proc-start-dir-user-writeable', 'reg-value-write-cert-change', 'reg-value-write-cert-change'), Frequenza: 3
Pattern: ('reg-value-write-cert-change', 'reg-value-write-cert-change', 'proc-start-dir-user-writeable'), Frequenza: 3
Pattern: ('proc-start-dir-user-writeable', 'reg-value-write-cert-change', 'reg-value-write-cert-change', 'reg-value-write-cert-change'), Frequenza: 3
Pattern: ('reg-value-write-cert-change', 'reg-value-write-cert-change', 'reg-value-write-cert-change', 'proc-start-dir-user-writeable'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-mavinject-inject-dll-into-running-process'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-mavinject-inject-dll-into-running-process'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-mavinject-inject-dll-into-running-process', 'proc-start-mavinject-inject-dll-into-running-process'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-mavinject-inject-dll-into-running-process', 'proc-start-mavinject-inject-dll-into-running-process'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-mavinject-inject-dll-into-running-process', 'proc-start-mavinject-inject-dll-into-running-process', 'proc-start-mavinject-inject-dll-into-running-process'), Frequenza: 3
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 3
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 3
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-suspicious-process-parents'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-suspicious-process-parents'), Frequenza: 3
Pattern: ('proc-start-hacktool-mimikatz-execution', 'proc-start-suspicious-program-names'), Frequenza: 3
Pattern: ('proc-start-hacktool-mimikatz-execution', 'proc-start-hacktool-mimikatz-execution', 'proc-start-suspicious-program-names'), Frequenza: 3
Pattern: ('proc-start-hacktool-mimikatz-execution', 'proc-start-hacktool-mimikatz-execution', 'proc-start-hacktool-mimikatz-execution', 'proc-start-suspicious-program-names'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-Windows-processes'), Frequenza: 3
Pattern: ('net-connect-80-443-non-browser', 'net-connect-Windows-processes', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-potentially-suspicious-powershell-child-processes'), Frequenza: 3
Pattern: ('net-connect-80-443-non-browser', 'net-connect-Windows-processes', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('proc-start-suspicious-reconnaissance-activity-via-gathernetworkinfo.vbs', 'proc-start-suspicious-reconnaissance-activity-via-gathernetworkinfo.vbs'), Frequenza: 3
Pattern: ('proc-start-potential-lsass-process-dump-via-procdump', 'proc-start-potential-lsass-process-dump-via-procdump'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 3
Pattern: ('proc-start-potential-meterpreter/cobaltstrike-activity', 'proc-start-potential-meterpreter/cobaltstrike-activity'), Frequenza: 3
In signature_patterns vediamo le sequenze di 3, 4 o 5 regole in ordine da quella più a quella meno frequente ripetutesi più volte durante i vari attacchi e che non compaiono mai tra le sequenze di falsi attacchi
With specified severity value¶
In [ ]:
result_pattern_inside_attack = analyzer.pattern_inside_attack(severity_value=severity_value)
result_pattern_inside_attack
MITRE ATT&CK IDs:
1-digit repetitions:
2-digits sequences:
('T1003.001', 'T1003.001'): 23
('T1055.001', 'T1055.001'): 3
('T1059', 'T1059.001'): 3
('T1003.002', 'T1003.002'): 2
('T1070.001', 'T1070.001'): 2
('T1003', 'T1036'): 2
('T1021.001', 'T1021.001'): 1
('T1053.005', 'T1053.005'): 1
('T1016', 'T1016'): 1
('T1059.001', 'T1059'): 1
('T1105', 'T1059.001'): 1
('T1070', 'T1070'): 1
3-digits sequences:
('T1482', 'T1482', 'T1482'): 21
('T1003.001', 'T1003.001', 'T1003.001'): 11
('T1059', 'T1003.001', 'T1003.001'): 4
('T1049', 'T1049', 'T1049'): 4
('T1027', 'T1027', 'T1027'): 3
('T1106', 'T1003.001', 'T1003.001'): 3
('T1112', 'T1112', 'T1562.001'): 2
('T1003.001', 'T1548', 'T1036'): 2
('T1003.001', 'T1003.001', 'T1059.001'): 2
('T1003.001', 'T1003.001', 'T1482'): 2
('T1059', 'T1059.001', 'T1105'): 2
('T1059', 'T1059.001', 'T1003.001'): 2
('T1003.001', 'T1003.001', 'T1059'): 2
('T1059.001', 'T1482', 'T1482'): 2
('T1105', 'T1482', 'T1482'): 1
('T1003.002', 'T1003.002', 'T1003.003'): 1
('T1003.001', 'T1003.001', 'T1543.003'): 1
('T1105', 'T1018', 'T1018'): 1
('T1059', 'T1059', 'T1482'): 1
('T1033', 'T1003.001', 'T1003.001'): 1
('T1548', 'T1105', 'T1003.001'): 1
('T1036.003', 'T1566.001', 'T1548'): 1
('T1059.001', 'T1105', 'T1003.001'): 1
('T1105', 'T1105', 'T1003.001'): 1
('T1106', 'T1059.001', 'T1482'): 1
('T1548', 'T1036', 'T1003.001'): 1
('T1055.001', 'T1055.001', 'T1218.013'): 1
('T1070', 'T1070', 'T1485'): 1
('T1059.001', 'T1059', 'T1059.001'): 1
('T1059.001', 'T1059', 'T1482'): 1
('T1036', 'T1036', 'T1003.001'): 1
('T1003.001', 'T1003.001', 'T1036'): 1
('T1003.001', 'T1059.001', 'T1059'): 1
('T1105', 'T1105', 'T1105'): 1
('T1105', 'T1543.003', 'T1003.001'): 1
('T1036.003', 'T1548', 'T1036'): 1
('T1036.003', 'T1548', 'T1566.001'): 1
('T1548', 'T1588.003', 'T1588.003'): 1
('T1059', 'T1059', 'T1059.001'): 1
('T1059', 'T1059.001', 'T1059'): 1
('T1018', 'T1018', 'T1018'): 1
('T1564.004', 'T1133', 'T1133'): 1
('T1059.001', 'T1555.004', 'T1218.011'): 1
('T1059.001', 'T1059', 'T1105'): 1
SIGNATURES:
1-digit repetitions:
2-digits sequences:
('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 23
('proc-start-mavinject-inject-dll-into-running-process', 'proc-start-mavinject-inject-dll-into-running-process'): 3
('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern'): 3
('proc-start-suspicious-eventlog-clear-or-configuration-change', 'proc-start-suspicious-eventlog-clear-or-configuration-change'): 2
('proc-start-potential-credential-dumping-attempt-using-new-networkprovider-cli', 'proc-start-process-memory-dump-via-comsvcs.dll'): 2
('proc-start-volumeshadowcopy-symlink-creation-via-mklink', 'proc-start-volumeshadowcopy-symlink-creation-via-mklink'): 1
('proc-start-potential-tampering-with-rdp-related-registry-keys-via-reg.exe', 'proc-start-potential-tampering-with-rdp-related-registry-keys-via-reg.exe'): 1
('proc-start-suspicious-command-patterns-in-scheduled-task-creation', 'proc-start-schtasks-creation-or-modification-with-system-privileges'): 1
('proc-start-copying-sensitive-files-with-credential-data', 'proc-start-copying-sensitive-files-with-credential-data'): 1
('proc-start-potential-recon-activity-via-nltest.exe', 'proc-start-potential-recon-activity-via-nltest.exe'): 1
('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-powershell-download-and-execution-cradles'): 1
('proc-start-suspicious-invoke-webrequest-execution', 'proc-start-potential-data-exfiltration-activity-via-commandline-tools'): 1
('proc-start-fsutil-suspicious-invocation', 'proc-start-fsutil-suspicious-invocation'): 1
3-digits sequences:
('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'): 21
('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 9
('proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution'): 4
('proc-start-base64-encoded-powershell-command-detected', 'proc-start-base64-encoded-powershell-command-detected', 'proc-start-base64-encoded-powershell-command-detected'): 3
('proc-start-potential-winapi-calls-via-commandline', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 3
('proc-start-reg-add-suspicious-paths', 'proc-start-reg-add-suspicious-paths', 'proc-start-reg-add-suspicious-paths'): 2
('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-dir-user-writeable', 'proc-start-system-file-execution-location-anomaly'): 2
('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-malicious-powershell-commandlets-processcreation'): 2
('proc-start-suspicious-program-names', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 2
('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-suspicious-target-names'): 2
('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 2
('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-powershell-download-and-execution-cradles'): 2
('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'): 2
('net-connect-80-443-non-browser', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'): 1
('proc-start-copying-sensitive-files-with-credential-data', 'proc-start-copying-sensitive-files-with-credential-data', 'proc-start-copying-sensitive-files-with-credential-data'): 1
('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'reg-key-create-service'): 1
('net-connect-80-443-non-browser', 'proc-start-pua-adfind-suspicious-execution', 'proc-start-pua-adfind-suspicious-execution'): 1
('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-program-names', 'proc-start-malicious-powershell-commandlets-processcreation'): 1
('proc-start-suspicious-whoami.exe-execution', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 1
('proc-start-dir-user-writeable', 'net-connect-80-443-non-browser', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 1
('proc-start-lol-binary-copied-from-system-directory', 'proc-start-suspicious-double-extension-file-execution', 'proc-start-dir-user-writeable'): 1
('proc-start-potential-data-exfiltration-activity-via-commandline-tools', 'proc-start-suspicious-invoke-webrequest-execution', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 1
('proc-start-hacktool-mimikatz-execution', 'proc-start-hacktool-mimikatz-execution', 'proc-start-hacktool-mimikatz-execution'): 1
('net-connect-80-443-non-browser', 'net-connect-80-443-non-browser', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 1
('proc-start-potential-winapi-calls-via-commandline', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-malicious-powershell-commandlets-processcreation'): 1
('proc-start-dir-user-writeable', 'proc-start-suspicious-process-parents', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 1
('proc-start-powershell-download-and-execution-cradles', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 1
('proc-start-mavinject-inject-dll-into-running-process', 'proc-start-mavinject-inject-dll-into-running-process', 'proc-start-mavinject-inject-dll-into-running-process'): 1
('proc-start-powershell-download-and-execution-cradles', 'proc-start-hacktool-mimikatz-execution', 'proc-start-hacktool-mimikatz-execution'): 1
('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-suspicious-powershell-download-and-execute-pattern'): 1
('proc-start-fsutil-suspicious-invocation', 'proc-start-fsutil-suspicious-invocation', 'proc-start-fsutil-suspicious-invocation'): 1
('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-potentially-suspicious-powershell-child-processes'): 1
('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-potentially-suspicious-powershell-child-processes'): 1
('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-malicious-powershell-commandlets-processcreation'): 1
('proc-start-potential-lsass-process-dump-via-procdump', 'proc-start-potential-lsass-process-dump-via-procdump', 'proc-start-potential-lsass-process-dump-via-procdump'): 1
('proc-start-lsass-dump-keyword-in-commandline', 'proc-start-lsass-dump-keyword-in-commandline', 'proc-start-process-memory-dump-via-comsvcs.dll'): 1
('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-lsass-dump-keyword-in-commandline'): 1
('proc-start-lsass-dump-keyword-in-commandline', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-powershell-download-and-execution-cradles'): 1
('net-connect-Windows-processes', 'net-connect-Windows-processes', 'net-connect-Windows-processes'): 1
('proc-start-suspicious-invoke-webrequest-execution', 'reg-key-create-service', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 1
('proc-start-lol-binary-copied-from-system-directory', 'proc-start-dir-user-writeable', 'proc-start-system-file-execution-location-anomaly'): 1
('proc-start-lol-binary-copied-from-system-directory', 'proc-start-dir-user-writeable', 'proc-start-suspicious-double-extension-file-execution'): 1
('proc-start-dir-user-writeable', 'reg-value-write-cert-change', 'reg-value-write-cert-change'): 1
('proc-start-suspicious-program-names', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern'): 1
('proc-start-suspicious-program-names', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-powershell-download-and-execution-cradles'): 1
('proc-start-renamed-adfind-execution', 'proc-start-renamed-adfind-execution', 'proc-start-renamed-adfind-execution'): 1
('proc-start-lolbas-alternate-data-streams', 'proc-start-suspicious-add-user-to-remote-desktop-users-group', 'proc-start-suspicious-add-user-to-remote-desktop-users-group'): 1
('proc-start-potentially-suspicious-powershell-child-processes', 'proc-start-suspicious-key-manager-access', 'proc-start-rundll32-execution-without-dll-file'): 1
('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-powershell-download-and-execution-cradles', 'net-connect-suspicious-target-names'): 1
In result_pattern_inside_attack vediamo:
- '1-digit repetitions' che corrisponde alle ripetizioni di mitre_attack.id e signature in testa agli attacchi con un numero massimo di 3 mitre o signature registrati;
- '2-digits sequences' che corrisponde alle sequenze di mitre_attack.id e signature in testa agli attacchi con un numero di mitre o signature compreso tra 4 e 5;
- '3-digits sequences' che corrisponde alle sequenze di mitre_attack.id e signature in testa agli attacchi con un numero di mitre o signature maggiore di 5 (non compreso).
Correlation Matrix¶
In [ ]:
CorrelationMatrixPlots.plot_correlation_matrix(result_df_Le, 'Correlation Matrix (Label Encoding)')
CorrelationMatrixPlots.plot_correlation_matrix_big(result_df_OH, 'Correlation Matrix (OneHot Encoding)')
ML¶
OneHot¶
In [ ]:
# Split data
X_train_OH, X_test_OH, y_train_OH, y_test_OH = PreprocessingTrainTestSplit.split_data(result_df_OH, "corrisponde_ad_attacco")
# Initial model training and evaluation
InitialTraining.train_and_evaluate_initial_models(X_train_OH, y_train_OH, X_test_OH, y_test_OH)
# Hyperparameter tuning
best_models_OH = HyperparameterTuning.tune_hyperparameters(X_train_OH, y_train_OH)
# Evaluate best models on test set
evaluator_OH = ModelEvaluator(best_models_OH)
evaluation_results_OH = evaluator_OH.evaluate_models(X_test_OH, y_test_OH)
# Train XGBoost model
AdvancedModels.train_xgboost(X_train_OH, y_train_OH, X_test_OH, y_test_OH)
# Train deep learning model
DeepLearningModel.train_deep_learning_model(X_train_OH, y_train_OH, X_test_OH, y_test_OH)
Decision Tree Classification Report:
precision recall f1-score support
0 0.83 0.82 0.82 742
1 0.92 0.93 0.92 1692
accuracy 0.89 2434
macro avg 0.87 0.87 0.87 2434
weighted avg 0.89 0.89 0.89 2434
AdaBoost Classification Report:
precision recall f1-score support
0 0.73 0.38 0.50 742
1 0.78 0.94 0.85 1692
accuracy 0.77 2434
macro avg 0.75 0.66 0.68 2434
weighted avg 0.76 0.77 0.74 2434
XGBoost Classification Report:
precision recall f1-score support
0 0.85 0.76 0.80 742
1 0.90 0.94 0.92 1692
accuracy 0.89 2434
macro avg 0.88 0.85 0.86 2434
weighted avg 0.88 0.89 0.88 2434
CatBoost Classification Report:
precision recall f1-score support
0 0.86 0.70 0.77 742
1 0.88 0.95 0.91 1692
accuracy 0.87 2434
macro avg 0.87 0.82 0.84 2434
weighted avg 0.87 0.87 0.87 2434
MLP Classification Report:
precision recall f1-score support
0 0.00 0.00 0.00 742
1 0.70 1.00 0.82 1692
accuracy 0.70 2434
macro avg 0.35 0.50 0.41 2434
weighted avg 0.48 0.70 0.57 2434
Quadratic Discriminant Analysis Classification Report:
precision recall f1-score support
0 0.33 0.97 0.50 742
1 0.92 0.15 0.26 1692
accuracy 0.40 2434
macro avg 0.63 0.56 0.38 2434
weighted avg 0.74 0.40 0.33 2434
Extra Trees Classification Report:
precision recall f1-score support
0 0.84 0.81 0.82 742
1 0.92 0.93 0.92 1692
accuracy 0.89 2434
macro avg 0.88 0.87 0.87 2434
weighted avg 0.89 0.89 0.89 2434
Best parameters for Random Forest: {'max_depth': None, 'min_samples_split': 5, 'n_estimators': 200}
Best F1-score: 0.9293148321221679
Best parameters for Gradient Boosting: {'learning_rate': 0.3, 'max_depth': 7, 'n_estimators': 200}
Best F1-score: 0.9303527123419139
Best parameters for Naive Bayes: {}
Best F1-score: 0.8124441534677572
Best parameters for KNN: {'knn__metric': 'manhattan', 'knn__n_neighbors': 3, 'knn__weights': 'distance'}
Best F1-score: 0.9702072431513097
Best parameters for Logistic Regression: {'logreg__C': 0.01, 'logreg__solver': 'lbfgs'}
Best F1-score: 0.8176609276895921
Random Forest Classification Report:
precision recall f1-score support
0 0.85 0.78 0.81 742
1 0.91 0.94 0.92 1692
accuracy 0.89 2434
macro avg 0.88 0.86 0.87 2434
weighted avg 0.89 0.89 0.89 2434
Gradient Boosting Classification Report:
precision recall f1-score support
0 0.85 0.80 0.83 742
1 0.91 0.94 0.93 1692
accuracy 0.90 2434
macro avg 0.88 0.87 0.88 2434
weighted avg 0.90 0.90 0.90 2434
Naive Bayes Classification Report:
precision recall f1-score support
0 1.00 0.00 0.00 742
1 0.70 1.00 0.82 1692
accuracy 0.70 2434
macro avg 0.85 0.50 0.41 2434
weighted avg 0.79 0.70 0.57 2434
KNN Classification Report:
precision recall f1-score support
0 0.94 0.94 0.94 742
1 0.97 0.97 0.97 1692
accuracy 0.96 2434
macro avg 0.96 0.96 0.96 2434
weighted avg 0.96 0.96 0.96 2434
Logistic Regression Classification Report:
precision recall f1-score support
0 0.65 0.23 0.34 742
1 0.74 0.95 0.83 1692
accuracy 0.73 2434
macro avg 0.69 0.59 0.58 2434
weighted avg 0.71 0.73 0.68 2434
[0] train-auc:0.76568 eval-auc:0.75568
[1] train-auc:0.78509 eval-auc:0.78014
[2] train-auc:0.81380 eval-auc:0.80464
[3] train-auc:0.82123 eval-auc:0.81147
[4] train-auc:0.82870 eval-auc:0.81905
[5] train-auc:0.83596 eval-auc:0.82512
[6] train-auc:0.84242 eval-auc:0.82579
[7] train-auc:0.84401 eval-auc:0.82637
[8] train-auc:0.84393 eval-auc:0.82365
[9] train-auc:0.84584 eval-auc:0.82448
[10] train-auc:0.85439 eval-auc:0.83049
[11] train-auc:0.85558 eval-auc:0.83124
[12] train-auc:0.85985 eval-auc:0.83781
[13] train-auc:0.85975 eval-auc:0.83772
[14] train-auc:0.85961 eval-auc:0.83680
[15] train-auc:0.86092 eval-auc:0.83931
[16] train-auc:0.86503 eval-auc:0.84223
[17] train-auc:0.86582 eval-auc:0.84304
[18] train-auc:0.86807 eval-auc:0.84528
[19] train-auc:0.86789 eval-auc:0.84410
[20] train-auc:0.86771 eval-auc:0.84374
[21] train-auc:0.87046 eval-auc:0.84721
[22] train-auc:0.87127 eval-auc:0.84681
[23] train-auc:0.87156 eval-auc:0.84762
[24] train-auc:0.87286 eval-auc:0.84939
[25] train-auc:0.87426 eval-auc:0.85127
[26] train-auc:0.87440 eval-auc:0.85208
[27] train-auc:0.87449 eval-auc:0.85242
[28] train-auc:0.87467 eval-auc:0.85237
[29] train-auc:0.87517 eval-auc:0.85280
[30] train-auc:0.87525 eval-auc:0.85315
[31] train-auc:0.87598 eval-auc:0.85362
[32] train-auc:0.87708 eval-auc:0.85380
[33] train-auc:0.87750 eval-auc:0.85367
[34] train-auc:0.87705 eval-auc:0.85308
[35] train-auc:0.87777 eval-auc:0.85385
[36] train-auc:0.87878 eval-auc:0.85431
[37] train-auc:0.87926 eval-auc:0.85441
[38] train-auc:0.88014 eval-auc:0.85497
[39] train-auc:0.88198 eval-auc:0.85692
[40] train-auc:0.88236 eval-auc:0.85709
[41] train-auc:0.88541 eval-auc:0.85928
[42] train-auc:0.88639 eval-auc:0.85992
[43] train-auc:0.88933 eval-auc:0.86267
[44] train-auc:0.89108 eval-auc:0.86487
[45] train-auc:0.89265 eval-auc:0.86630
[46] train-auc:0.89272 eval-auc:0.86648
[47] train-auc:0.89394 eval-auc:0.86831
[48] train-auc:0.89400 eval-auc:0.86899
[49] train-auc:0.89482 eval-auc:0.86904
[50] train-auc:0.89536 eval-auc:0.86982
[51] train-auc:0.89657 eval-auc:0.87147
[52] train-auc:0.89772 eval-auc:0.87317
[53] train-auc:0.89878 eval-auc:0.87395
[54] train-auc:0.89910 eval-auc:0.87391
[55] train-auc:0.90012 eval-auc:0.87415
[56] train-auc:0.90029 eval-auc:0.87452
[57] train-auc:0.90168 eval-auc:0.87603
[58] train-auc:0.90177 eval-auc:0.87603
[59] train-auc:0.90191 eval-auc:0.87605
[60] train-auc:0.90195 eval-auc:0.87588
[61] train-auc:0.90365 eval-auc:0.87781
[62] train-auc:0.90453 eval-auc:0.87917
[63] train-auc:0.90462 eval-auc:0.87910
[64] train-auc:0.90582 eval-auc:0.88033
[65] train-auc:0.90571 eval-auc:0.88039
[66] train-auc:0.90583 eval-auc:0.88033
[67] train-auc:0.90655 eval-auc:0.88126
[68] train-auc:0.90841 eval-auc:0.88286
[69] train-auc:0.90900 eval-auc:0.88307
[70] train-auc:0.90936 eval-auc:0.88328
[71] train-auc:0.91196 eval-auc:0.88701
[72] train-auc:0.91258 eval-auc:0.88747
[73] train-auc:0.91345 eval-auc:0.88829
[74] train-auc:0.91344 eval-auc:0.88828
[75] train-auc:0.91430 eval-auc:0.88971
[76] train-auc:0.91470 eval-auc:0.89033
[77] train-auc:0.91515 eval-auc:0.89072
[78] train-auc:0.91604 eval-auc:0.89181
[79] train-auc:0.91712 eval-auc:0.89247
[80] train-auc:0.91727 eval-auc:0.89246
[81] train-auc:0.91736 eval-auc:0.89262
[82] train-auc:0.91885 eval-auc:0.89335
[83] train-auc:0.91901 eval-auc:0.89367
[84] train-auc:0.91967 eval-auc:0.89454
[85] train-auc:0.92011 eval-auc:0.89482
[86] train-auc:0.92048 eval-auc:0.89540
[87] train-auc:0.92071 eval-auc:0.89547
[88] train-auc:0.92058 eval-auc:0.89536
[89] train-auc:0.92085 eval-auc:0.89558
[90] train-auc:0.92119 eval-auc:0.89594
[91] train-auc:0.92189 eval-auc:0.89624
[92] train-auc:0.92221 eval-auc:0.89677
[93] train-auc:0.92262 eval-auc:0.89739
[94] train-auc:0.92280 eval-auc:0.89745
[95] train-auc:0.92309 eval-auc:0.89750
[96] train-auc:0.92342 eval-auc:0.89833
[97] train-auc:0.92387 eval-auc:0.89863
[98] train-auc:0.92415 eval-auc:0.89927
[99] train-auc:0.92445 eval-auc:0.89960
[100] train-auc:0.92495 eval-auc:0.90014
[101] train-auc:0.92503 eval-auc:0.90051
[102] train-auc:0.92513 eval-auc:0.90049
[103] train-auc:0.92526 eval-auc:0.90049
[104] train-auc:0.92556 eval-auc:0.90103
[105] train-auc:0.92609 eval-auc:0.90137
[106] train-auc:0.92615 eval-auc:0.90189
[107] train-auc:0.92703 eval-auc:0.90309
[108] train-auc:0.92714 eval-auc:0.90317
[109] train-auc:0.92722 eval-auc:0.90326
[110] train-auc:0.92729 eval-auc:0.90345
[111] train-auc:0.92728 eval-auc:0.90352
[112] train-auc:0.92736 eval-auc:0.90352
[113] train-auc:0.92747 eval-auc:0.90363
[114] train-auc:0.92779 eval-auc:0.90393
[115] train-auc:0.92812 eval-auc:0.90401
[116] train-auc:0.92823 eval-auc:0.90422
[117] train-auc:0.92854 eval-auc:0.90479
[118] train-auc:0.92866 eval-auc:0.90494
[119] train-auc:0.92877 eval-auc:0.90499
[120] train-auc:0.92887 eval-auc:0.90546
[121] train-auc:0.92911 eval-auc:0.90595
[122] train-auc:0.92931 eval-auc:0.90570
[123] train-auc:0.92944 eval-auc:0.90570
[124] train-auc:0.92953 eval-auc:0.90573
[125] train-auc:0.92965 eval-auc:0.90585
[126] train-auc:0.92991 eval-auc:0.90608
[127] train-auc:0.93009 eval-auc:0.90620
[128] train-auc:0.93044 eval-auc:0.90650
[129] train-auc:0.93096 eval-auc:0.90689
[130] train-auc:0.93149 eval-auc:0.90707
[131] train-auc:0.93181 eval-auc:0.90765
[132] train-auc:0.93176 eval-auc:0.90762
[133] train-auc:0.93210 eval-auc:0.90799
[134] train-auc:0.93254 eval-auc:0.90857
[135] train-auc:0.93272 eval-auc:0.90862
[136] train-auc:0.93315 eval-auc:0.90903
[137] train-auc:0.93332 eval-auc:0.90915
[138] train-auc:0.93359 eval-auc:0.90945
[139] train-auc:0.93361 eval-auc:0.90933
[140] train-auc:0.93374 eval-auc:0.90940
[141] train-auc:0.93379 eval-auc:0.90956
[142] train-auc:0.93415 eval-auc:0.90971
[143] train-auc:0.93411 eval-auc:0.90972
[144] train-auc:0.93430 eval-auc:0.90990
[145] train-auc:0.93441 eval-auc:0.91024
[146] train-auc:0.93460 eval-auc:0.91034
[147] train-auc:0.93499 eval-auc:0.91066
[148] train-auc:0.93501 eval-auc:0.91070
[149] train-auc:0.93519 eval-auc:0.91096
[150] train-auc:0.93522 eval-auc:0.91110
[151] train-auc:0.93547 eval-auc:0.91107
[152] train-auc:0.93566 eval-auc:0.91140
[153] train-auc:0.93588 eval-auc:0.91196
[154] train-auc:0.93620 eval-auc:0.91223
[155] train-auc:0.93649 eval-auc:0.91246
[156] train-auc:0.93680 eval-auc:0.91283
[157] train-auc:0.93698 eval-auc:0.91308
[158] train-auc:0.93710 eval-auc:0.91321
[159] train-auc:0.93732 eval-auc:0.91319
[160] train-auc:0.93751 eval-auc:0.91363
[161] train-auc:0.93779 eval-auc:0.91400
[162] train-auc:0.93781 eval-auc:0.91406
[163] train-auc:0.93816 eval-auc:0.91435
[164] train-auc:0.93814 eval-auc:0.91454
[165] train-auc:0.93855 eval-auc:0.91460
[166] train-auc:0.93869 eval-auc:0.91474
[167] train-auc:0.93871 eval-auc:0.91463
[168] train-auc:0.93875 eval-auc:0.91475
[169] train-auc:0.93891 eval-auc:0.91469
[170] train-auc:0.93918 eval-auc:0.91499
[171] train-auc:0.93920 eval-auc:0.91498
[172] train-auc:0.93925 eval-auc:0.91506
[173] train-auc:0.93958 eval-auc:0.91559
[174] train-auc:0.93975 eval-auc:0.91583
[175] train-auc:0.94013 eval-auc:0.91643
[176] train-auc:0.94031 eval-auc:0.91648
[177] train-auc:0.94046 eval-auc:0.91661
[178] train-auc:0.94069 eval-auc:0.91685
[179] train-auc:0.94077 eval-auc:0.91694
[180] train-auc:0.94079 eval-auc:0.91681
[181] train-auc:0.94104 eval-auc:0.91743
[182] train-auc:0.94123 eval-auc:0.91755
[183] train-auc:0.94148 eval-auc:0.91758
[184] train-auc:0.94162 eval-auc:0.91771
[185] train-auc:0.94185 eval-auc:0.91790
[186] train-auc:0.94257 eval-auc:0.91867
[187] train-auc:0.94259 eval-auc:0.91876
[188] train-auc:0.94256 eval-auc:0.91886
[189] train-auc:0.94272 eval-auc:0.91908
[190] train-auc:0.94276 eval-auc:0.91905
[191] train-auc:0.94282 eval-auc:0.91907
[192] train-auc:0.94316 eval-auc:0.91927
[193] train-auc:0.94332 eval-auc:0.91919
[194] train-auc:0.94362 eval-auc:0.91929
[195] train-auc:0.94383 eval-auc:0.91968
[196] train-auc:0.94381 eval-auc:0.91964
[197] train-auc:0.94388 eval-auc:0.91969
[198] train-auc:0.94392 eval-auc:0.91958
[199] train-auc:0.94394 eval-auc:0.91959
Accuracy: 84.22%
ROC AUC: 0.92
precision recall f1-score support
0 0.71 0.81 0.76 742
1 0.91 0.86 0.88 1692
accuracy 0.84 2434
macro avg 0.81 0.83 0.82 2434
weighted avg 0.85 0.84 0.84 2434
Epoch 1/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - accuracy: 0.5643 - loss: 1782025.7500
Epoch 2/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5800 - loss: 1308241.8750
Epoch 3/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5599 - loss: 1015836.6875
Epoch 4/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5627 - loss: 774538.5625
Epoch 5/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5730 - loss: 1049485.7500
Epoch 6/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.5661 - loss: 575765.3125
Epoch 7/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5711 - loss: 1126287.0000
Epoch 8/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.5852 - loss: 982926.8125
Epoch 9/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5663 - loss: 1464671.3750
Epoch 10/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.5636 - loss: 1053796.8750
Test Accuracy: 0.6951519846916199
77/77 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step
Classification Report for Deep Learning Model:
precision recall f1-score support
0 0.00 0.00 0.00 742
1 0.70 1.00 0.82 1692
accuracy 0.70 2434
macro avg 0.35 0.50 0.41 2434
weighted avg 0.48 0.70 0.57 2434
Out[ ]:
<Sequential name=sequential, built=True>
Label¶
In [ ]:
# Split data
X_train_Le, X_test_Le, y_train_Le, y_test_Le = PreprocessingTrainTestSplit.split_data(result_df_Le, "corrisponde_ad_attacco")
# Initial model training and evaluation
InitialTraining.train_and_evaluate_initial_models(X_train_Le, y_train_Le, X_test_Le, y_test_Le)
# Hyperparameter tuning
best_models_Le = HyperparameterTuning.tune_hyperparameters(X_train_Le, y_train_Le)
# Evaluate best models on test set
evaluator_Le = ModelEvaluator(best_models_Le)
evaluation_results_Le = evaluator_Le.evaluate_models(X_test_Le, y_test_Le)
# Train XGBoost model
AdvancedModels.train_xgboost(X_train_Le, y_train_Le, X_test_Le, y_test_Le)
# Train deep learning model
DeepLearningModel.train_deep_learning_model(X_train_Le, y_train_Le, X_test_Le, y_test_Le)
Decision Tree Classification Report:
precision recall f1-score support
0 0.83 0.82 0.83 742
1 0.92 0.93 0.92 1692
accuracy 0.89 2434
macro avg 0.88 0.88 0.88 2434
weighted avg 0.89 0.89 0.89 2434
AdaBoost Classification Report:
precision recall f1-score support
0 0.74 0.29 0.42 742
1 0.75 0.96 0.84 1692
accuracy 0.75 2434
macro avg 0.75 0.62 0.63 2434
weighted avg 0.75 0.75 0.71 2434
XGBoost Classification Report:
precision recall f1-score support
0 0.86 0.77 0.81 742
1 0.90 0.95 0.92 1692
accuracy 0.89 2434
macro avg 0.88 0.86 0.87 2434
weighted avg 0.89 0.89 0.89 2434
CatBoost Classification Report:
precision recall f1-score support
0 0.86 0.71 0.78 742
1 0.88 0.95 0.91 1692
accuracy 0.88 2434
macro avg 0.87 0.83 0.85 2434
weighted avg 0.88 0.88 0.87 2434
MLP Classification Report:
precision recall f1-score support
0 0.00 0.00 0.00 742
1 0.70 1.00 0.82 1692
accuracy 0.70 2434
macro avg 0.35 0.50 0.41 2434
weighted avg 0.48 0.70 0.57 2434
Quadratic Discriminant Analysis Classification Report:
precision recall f1-score support
0 0.49 0.19 0.28 742
1 0.72 0.91 0.81 1692
accuracy 0.69 2434
macro avg 0.61 0.55 0.54 2434
weighted avg 0.65 0.69 0.64 2434
Extra Trees Classification Report:
precision recall f1-score support
0 0.84 0.81 0.82 742
1 0.92 0.93 0.92 1692
accuracy 0.89 2434
macro avg 0.88 0.87 0.87 2434
weighted avg 0.89 0.89 0.89 2434
Best parameters for Random Forest: {'max_depth': 20, 'min_samples_split': 2, 'n_estimators': 300}
Best F1-score: 0.9328604085098989
Best parameters for Gradient Boosting: {'learning_rate': 0.3, 'max_depth': 7, 'n_estimators': 300}
Best F1-score: 0.9324297787118544
Best parameters for Naive Bayes: {}
Best F1-score: 0.8117828645683514
Best parameters for KNN: {'knn__metric': 'manhattan', 'knn__n_neighbors': 9, 'knn__weights': 'distance'}
Best F1-score: 0.9771909225408988
Best parameters for Logistic Regression: {'logreg__C': 0.01, 'logreg__solver': 'liblinear'}
Best F1-score: 0.8097576181622838
Random Forest Classification Report:
precision recall f1-score support
0 0.87 0.79 0.83 742
1 0.91 0.95 0.93 1692
accuracy 0.90 2434
macro avg 0.89 0.87 0.88 2434
weighted avg 0.90 0.90 0.90 2434
Gradient Boosting Classification Report:
precision recall f1-score support
0 0.86 0.81 0.83 742
1 0.92 0.94 0.93 1692
accuracy 0.90 2434
macro avg 0.89 0.88 0.88 2434
weighted avg 0.90 0.90 0.90 2434
Naive Bayes Classification Report:
precision recall f1-score support
0 0.00 0.00 0.00 742
1 0.70 1.00 0.82 1692
accuracy 0.70 2434
macro avg 0.35 0.50 0.41 2434
weighted avg 0.48 0.70 0.57 2434
KNN Classification Report:
precision recall f1-score support
0 0.95 0.95 0.95 742
1 0.98 0.98 0.98 1692
accuracy 0.97 2434
macro avg 0.96 0.96 0.96 2434
weighted avg 0.97 0.97 0.97 2434
Logistic Regression Classification Report:
precision recall f1-score support
0 0.49 0.02 0.05 742
1 0.70 0.99 0.82 1692
accuracy 0.69 2434
macro avg 0.59 0.51 0.43 2434
weighted avg 0.63 0.69 0.58 2434
[0] train-auc:0.79334 eval-auc:0.76224
[1] train-auc:0.81766 eval-auc:0.77658
[2] train-auc:0.84054 eval-auc:0.81116
[3] train-auc:0.84337 eval-auc:0.81958
[4] train-auc:0.85193 eval-auc:0.82875
[5] train-auc:0.85569 eval-auc:0.83242
[6] train-auc:0.85806 eval-auc:0.83798
[7] train-auc:0.85961 eval-auc:0.83771
[8] train-auc:0.86074 eval-auc:0.84046
[9] train-auc:0.86104 eval-auc:0.84106
[10] train-auc:0.86334 eval-auc:0.84423
[11] train-auc:0.86248 eval-auc:0.84302
[12] train-auc:0.86743 eval-auc:0.84778
[13] train-auc:0.86726 eval-auc:0.84784
[14] train-auc:0.86811 eval-auc:0.84833
[15] train-auc:0.87042 eval-auc:0.84889
[16] train-auc:0.87092 eval-auc:0.84964
[17] train-auc:0.87668 eval-auc:0.85663
[18] train-auc:0.87711 eval-auc:0.85868
[19] train-auc:0.88037 eval-auc:0.86024
[20] train-auc:0.88371 eval-auc:0.86271
[21] train-auc:0.88373 eval-auc:0.86344
[22] train-auc:0.88479 eval-auc:0.86438
[23] train-auc:0.88479 eval-auc:0.86406
[24] train-auc:0.88528 eval-auc:0.86467
[25] train-auc:0.88595 eval-auc:0.86517
[26] train-auc:0.88877 eval-auc:0.86710
[27] train-auc:0.89219 eval-auc:0.87088
[28] train-auc:0.89245 eval-auc:0.87072
[29] train-auc:0.89273 eval-auc:0.87067
[30] train-auc:0.89557 eval-auc:0.87294
[31] train-auc:0.89553 eval-auc:0.87270
[32] train-auc:0.89798 eval-auc:0.87413
[33] train-auc:0.89797 eval-auc:0.87401
[34] train-auc:0.89818 eval-auc:0.87379
[35] train-auc:0.89845 eval-auc:0.87426
[36] train-auc:0.89928 eval-auc:0.87521
[37] train-auc:0.89979 eval-auc:0.87513
[38] train-auc:0.90022 eval-auc:0.87579
[39] train-auc:0.90066 eval-auc:0.87637
[40] train-auc:0.90080 eval-auc:0.87641
[41] train-auc:0.90188 eval-auc:0.87830
[42] train-auc:0.90250 eval-auc:0.87869
[43] train-auc:0.90296 eval-auc:0.87945
[44] train-auc:0.90451 eval-auc:0.88107
[45] train-auc:0.90589 eval-auc:0.88192
[46] train-auc:0.90624 eval-auc:0.88193
[47] train-auc:0.90653 eval-auc:0.88236
[48] train-auc:0.90672 eval-auc:0.88223
[49] train-auc:0.90815 eval-auc:0.88298
[50] train-auc:0.90867 eval-auc:0.88361
[51] train-auc:0.91014 eval-auc:0.88527
[52] train-auc:0.91112 eval-auc:0.88603
[53] train-auc:0.91142 eval-auc:0.88596
[54] train-auc:0.91238 eval-auc:0.88688
[55] train-auc:0.91404 eval-auc:0.88883
[56] train-auc:0.91422 eval-auc:0.88939
[57] train-auc:0.91615 eval-auc:0.89180
[58] train-auc:0.91614 eval-auc:0.89193
[59] train-auc:0.91631 eval-auc:0.89192
[60] train-auc:0.91667 eval-auc:0.89244
[61] train-auc:0.91721 eval-auc:0.89352
[62] train-auc:0.91719 eval-auc:0.89298
[63] train-auc:0.91771 eval-auc:0.89370
[64] train-auc:0.91808 eval-auc:0.89468
[65] train-auc:0.91917 eval-auc:0.89548
[66] train-auc:0.91933 eval-auc:0.89552
[67] train-auc:0.91923 eval-auc:0.89458
[68] train-auc:0.91991 eval-auc:0.89583
[69] train-auc:0.92107 eval-auc:0.89735
[70] train-auc:0.92133 eval-auc:0.89744
[71] train-auc:0.92176 eval-auc:0.89795
[72] train-auc:0.92252 eval-auc:0.89754
[73] train-auc:0.92309 eval-auc:0.89765
[74] train-auc:0.92366 eval-auc:0.89832
[75] train-auc:0.92440 eval-auc:0.89913
[76] train-auc:0.92480 eval-auc:0.89989
[77] train-auc:0.92596 eval-auc:0.90128
[78] train-auc:0.92635 eval-auc:0.90128
[79] train-auc:0.92658 eval-auc:0.90152
[80] train-auc:0.92752 eval-auc:0.90224
[81] train-auc:0.92787 eval-auc:0.90313
[82] train-auc:0.92836 eval-auc:0.90381
[83] train-auc:0.92877 eval-auc:0.90440
[84] train-auc:0.92912 eval-auc:0.90477
[85] train-auc:0.92965 eval-auc:0.90536
[86] train-auc:0.93076 eval-auc:0.90641
[87] train-auc:0.93093 eval-auc:0.90658
[88] train-auc:0.93102 eval-auc:0.90709
[89] train-auc:0.93155 eval-auc:0.90742
[90] train-auc:0.93210 eval-auc:0.90740
[91] train-auc:0.93216 eval-auc:0.90736
[92] train-auc:0.93249 eval-auc:0.90757
[93] train-auc:0.93243 eval-auc:0.90747
[94] train-auc:0.93271 eval-auc:0.90764
[95] train-auc:0.93319 eval-auc:0.90835
[96] train-auc:0.93395 eval-auc:0.90951
[97] train-auc:0.93431 eval-auc:0.90990
[98] train-auc:0.93459 eval-auc:0.91070
[99] train-auc:0.93522 eval-auc:0.91101
[100] train-auc:0.93613 eval-auc:0.91178
[101] train-auc:0.93642 eval-auc:0.91206
[102] train-auc:0.93683 eval-auc:0.91230
[103] train-auc:0.93709 eval-auc:0.91254
[104] train-auc:0.93798 eval-auc:0.91300
[105] train-auc:0.93803 eval-auc:0.91307
[106] train-auc:0.93834 eval-auc:0.91325
[107] train-auc:0.93872 eval-auc:0.91383
[108] train-auc:0.93893 eval-auc:0.91400
[109] train-auc:0.93906 eval-auc:0.91434
[110] train-auc:0.93915 eval-auc:0.91453
[111] train-auc:0.93935 eval-auc:0.91479
[112] train-auc:0.93943 eval-auc:0.91494
[113] train-auc:0.93953 eval-auc:0.91505
[114] train-auc:0.93975 eval-auc:0.91517
[115] train-auc:0.93991 eval-auc:0.91551
[116] train-auc:0.94008 eval-auc:0.91554
[117] train-auc:0.94037 eval-auc:0.91589
[118] train-auc:0.94067 eval-auc:0.91604
[119] train-auc:0.94101 eval-auc:0.91609
[120] train-auc:0.94134 eval-auc:0.91641
[121] train-auc:0.94142 eval-auc:0.91639
[122] train-auc:0.94153 eval-auc:0.91639
[123] train-auc:0.94160 eval-auc:0.91637
[124] train-auc:0.94180 eval-auc:0.91667
[125] train-auc:0.94189 eval-auc:0.91678
[126] train-auc:0.94213 eval-auc:0.91702
[127] train-auc:0.94225 eval-auc:0.91730
[128] train-auc:0.94255 eval-auc:0.91753
[129] train-auc:0.94312 eval-auc:0.91797
[130] train-auc:0.94340 eval-auc:0.91831
[131] train-auc:0.94359 eval-auc:0.91820
[132] train-auc:0.94417 eval-auc:0.91884
[133] train-auc:0.94461 eval-auc:0.91924
[134] train-auc:0.94459 eval-auc:0.91917
[135] train-auc:0.94489 eval-auc:0.91938
[136] train-auc:0.94523 eval-auc:0.92006
[137] train-auc:0.94528 eval-auc:0.91989
[138] train-auc:0.94540 eval-auc:0.91992
[139] train-auc:0.94563 eval-auc:0.92026
[140] train-auc:0.94597 eval-auc:0.92047
[141] train-auc:0.94608 eval-auc:0.92060
[142] train-auc:0.94625 eval-auc:0.92048
[143] train-auc:0.94644 eval-auc:0.92079
[144] train-auc:0.94675 eval-auc:0.92112
[145] train-auc:0.94695 eval-auc:0.92113
[146] train-auc:0.94720 eval-auc:0.92123
[147] train-auc:0.94731 eval-auc:0.92143
[148] train-auc:0.94732 eval-auc:0.92154
[149] train-auc:0.94743 eval-auc:0.92163
[150] train-auc:0.94770 eval-auc:0.92170
[151] train-auc:0.94790 eval-auc:0.92184
[152] train-auc:0.94822 eval-auc:0.92199
[153] train-auc:0.94831 eval-auc:0.92204
[154] train-auc:0.94838 eval-auc:0.92190
[155] train-auc:0.94873 eval-auc:0.92237
[156] train-auc:0.94877 eval-auc:0.92255
[157] train-auc:0.94908 eval-auc:0.92252
[158] train-auc:0.94954 eval-auc:0.92304
[159] train-auc:0.94970 eval-auc:0.92331
[160] train-auc:0.94983 eval-auc:0.92343
[161] train-auc:0.94995 eval-auc:0.92364
[162] train-auc:0.95004 eval-auc:0.92371
[163] train-auc:0.95040 eval-auc:0.92383
[164] train-auc:0.95075 eval-auc:0.92393
[165] train-auc:0.95106 eval-auc:0.92416
[166] train-auc:0.95104 eval-auc:0.92407
[167] train-auc:0.95124 eval-auc:0.92425
[168] train-auc:0.95132 eval-auc:0.92417
[169] train-auc:0.95149 eval-auc:0.92433
[170] train-auc:0.95184 eval-auc:0.92472
[171] train-auc:0.95214 eval-auc:0.92510
[172] train-auc:0.95246 eval-auc:0.92538
[173] train-auc:0.95249 eval-auc:0.92552
[174] train-auc:0.95264 eval-auc:0.92563
[175] train-auc:0.95280 eval-auc:0.92571
[176] train-auc:0.95291 eval-auc:0.92595
[177] train-auc:0.95298 eval-auc:0.92620
[178] train-auc:0.95328 eval-auc:0.92642
[179] train-auc:0.95334 eval-auc:0.92661
[180] train-auc:0.95344 eval-auc:0.92680
[181] train-auc:0.95357 eval-auc:0.92717
[182] train-auc:0.95378 eval-auc:0.92726
[183] train-auc:0.95396 eval-auc:0.92724
[184] train-auc:0.95418 eval-auc:0.92745
[185] train-auc:0.95438 eval-auc:0.92752
[186] train-auc:0.95458 eval-auc:0.92769
[187] train-auc:0.95457 eval-auc:0.92783
[188] train-auc:0.95457 eval-auc:0.92784
[189] train-auc:0.95467 eval-auc:0.92797
[190] train-auc:0.95478 eval-auc:0.92784
[191] train-auc:0.95491 eval-auc:0.92803
[192] train-auc:0.95515 eval-auc:0.92843
[193] train-auc:0.95516 eval-auc:0.92822
[194] train-auc:0.95535 eval-auc:0.92837
[195] train-auc:0.95552 eval-auc:0.92836
[196] train-auc:0.95565 eval-auc:0.92862
[197] train-auc:0.95562 eval-auc:0.92855
[198] train-auc:0.95583 eval-auc:0.92859
[199] train-auc:0.95584 eval-auc:0.92857
Accuracy: 85.95%
ROC AUC: 0.93
precision recall f1-score support
0 0.74 0.83 0.78 742
1 0.92 0.87 0.90 1692
accuracy 0.86 2434
macro avg 0.83 0.85 0.84 2434
weighted avg 0.87 0.86 0.86 2434
Epoch 1/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 2s 1ms/step - accuracy: 0.5767 - loss: 2932992.5000
Epoch 2/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5477 - loss: 2713778.0000
Epoch 3/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5695 - loss: 2416932.2500
Epoch 4/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5606 - loss: 1846083.6250
Epoch 5/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5566 - loss: 2680881.0000
Epoch 6/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5673 - loss: 2293997.5000
Epoch 7/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.5547 - loss: 1850834.0000
Epoch 8/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5722 - loss: 1339577.6250
Epoch 9/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5605 - loss: 2254119.0000
Epoch 10/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5698 - loss: 2419570.0000
Test Accuracy: 0.30484798550605774
77/77 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step
Classification Report for Deep Learning Model:
precision recall f1-score support
0 0.30 1.00 0.47 742
1 0.00 0.00 0.00 1692
accuracy 0.30 2434
macro avg 0.15 0.50 0.23 2434
weighted avg 0.09 0.30 0.14 2434
Out[ ]:
<Sequential name=sequential_1, built=True>
In [ ]:
evaluator_OH.print_best_model('OneHot Encoder')
evaluator_Le.print_best_model('Label Encoder')
Dopo la codifica con OneHot Encoder il modello migliore è stato KNN con lo score di 0.9559 Dopo la codifica con Label Encoder il modello migliore è stato KNN con lo score di 0.9612